ANNENBERG 


BROWN UNIVERSITY 


=Xo NYA) a dl ale) ag-) ol-1am \ (oor Oa be, 


Sustaining a Sense of Success: The 


Importance of Teacher Working Conditions 
During the COVID-19 Pandemic 


Matthew A. Kraft Nicole S. Simon Melissa Arnold Lyon 
Brown University City University of New York Brown University 


COVID-19 shuttered schools across the United States, upending traditional approaches to 
education. We examine teachers’ experiences during emergency remote teaching in the spring of 
2020 using responses to a working conditions survey from a sample of 7,841 teachers across 206 
schools and 9 states. Teachers reported a range of challenges related to engaging students in remote 
learning and balancing their professional and personal responsibilities. Teachers in high-poverty 
and majority Black schools perceived these challenges to be the most severe, suggesting the 
pandemic further increased existing educational inequities. Using data from both pre-post and 
retrospective surveys, we find that the pandemic and pivot to emergency remote teaching resulted 
in a sudden, large drop in teachers’ sense of success. We also demonstrate how supportive working 
conditions in schools played a critical role in helping teachers to sustain their sense of success. 
Teachers who could depend on their district and school-based leadership for strong communication, 
targeted training, meaningful collaboration, fair expectations, and recognition of their efforts were 
least likely to experience declines in their sense of success. 


VERSION: August 2020 


Suggested citation: Kraft, Matthew A., Nicole S. Simon, and Melissa Arnold Lyon. (2020). Sustaining a Sense of 
Success: The Importance of Teacher Working Conditions During the COVID-19 Pandemic . (EdWorkingPaper: 
20-279). Retrieved from Annenberg Institute at Brown University: https://doi.org/10.26300/35nj-v890 


Sustaining a Sense of Success: The Importance of Teacher Working Conditions During the 
COVID-19 Pandemic 


Matthew A. Kraft 
Brown University 


Nicole S. Simon 
City University of New York 


Melissa Arnold Lyon 
Brown University 


August 2020 


Abstract 


COVID-19 shuttered schools across the United States, upending traditional approaches to 
education. We examine teachers’ experiences during emergency remote teaching in the spring of 
2020 using responses to a working conditions survey from a sample of 7,841 teachers across 206 
schools and 9 states. Teachers reported a range of challenges related to engaging students in 
remote learning and balancing their professional and personal responsibilities. Teachers in high- 
poverty and majority Black schools perceived these challenges to be the most severe, suggesting 
the pandemic further increased existing educational inequities. Using data from both pre-post 
and retrospective surveys, we find that the pandemic and pivot to emergency remote teaching 
resulted in a sudden, large drop in teachers’ sense of success. We also demonstrate how 
supportive working conditions in schools played a critical role in helping teachers to sustain their 
sense of success. Teachers who could depend on their district and school-based leadership for 
strong communication, targeted training, meaningful collaboration, fair expectations, and 
recognition of their efforts were least likely to experience declines in their sense of success. 


Keywords: Teachers, Working Conditions, COVID-19, Student Engagement, Organizational 
Change, School Leadership, Equity 


Correspondence regarding the paper can be sent to Matthew Kraft at mkraft@brown.edu; Brown University, PO 
Box 1938, Providence RI, 02912. We are grateful to Henry Wellington, Kylie Davis and the Upbeat Team for their 
support of this research and to Marlene Almanzar who provided outstanding research assistance. Matthew Kraft is a 
paid consultant for Upbeat and serves in the role of Director of Research. Nicole Simon is a paid consultant for 
Upbeat. 


Sustaining a Sense of Success: The Importance of Teacher Working Conditions During the 
COVID-19 Pandemic 
Introduction 

Public schools have long been characterized as inflexible and bureaucratic organizations 
where teachers’ independence and autonomy are prized (Lortie, 1975). Teachers often work in 
isolation, learning from experience and refining their pedagogical practices in traditional 
classroom settings that have changed little over time. This schooling system functions, albeit 
imperfectly, during quiescent times. However, the limitations of this siloed, static culture are 
exposed when schools experience rapid organizational change. Now is such a time. 

The public health crisis caused by the COVID-19 virus shuttered schools across the 
United States in March, upending traditional approaches to education. Schools and teachers were 
forced to pivot to fully remote instruction with virtually no warning or preparation time. This 
sudden and total change in how teachers delivered instruction, combined with the health threats 
and economic consequences of the pandemic, created a uniquely stressful and demanding 
context for teachers’ work. 

In this paper, we examine the challenges teachers faced while working from home at the 
start of the pandemic, between March and June 2020, and we explore the role that working 
conditions played in supporting their sense of success in this new remote setting. Doing so can 
inform schools’ efforts to support teachers during what will be an extended period of non- 
traditional teaching. It can also inform broader efforts to institute lasting changes to how we 
organize and deliver public education. We focus our analyses around four central research 


questions (RQ): 


RQI1; What challenges did the pandemic and sudden change to emergency remote 
teaching present for teachers ? 


RQ2: How did these challenges differ across teacher and school characteristics ? 


RQ3: How did the pandemic and transition to emergency remote teaching affect 
teachers’ sense of success? 


RQ4: What role did working conditions play in sustaining teachers’ sense of success 

during the transition to emergency remote teaching? 

We explore these questions using two waves of a working conditions survey from the fall 
and spring of the 2019-2020 school year. The surveys were administered to a large and diverse 
sample of teachers by Upbeat, a firm that supports school districts with teacher retention. For the 
Spring administration, we used a retrospective survey design by asking teachers to answer items 
based on their experiences prior to the pandemic and to respond to additional questions 
specifically about teaching during the pandemic. Over 7,800 teachers working across 10 districts 
and 5 charter school networks completed the spring survey. Together, data from the fall and 
spring of the 2019-2020 school year allow us to conduct a range of descriptive analyses and to 
employ longitudinal growth models and value-added models to explore how teachers’ sense of 
success changed over time and the degree to which working conditions moderated these changes. 

Our findings suggest that the sudden move to remote teaching created substantial 
challenges for teachers’ work and limited the degree to which students could engage in learning. 
Teachers at every career phase struggled with the shift to remote teaching, but often in different 
ways. Mid-career teachers — those most likely to have children at home — struggled to balance 
professional responsibilities with obligations at home. Veteran teachers were much more likely 


to report being uncomfortable using the technological tools required for teaching remotely. 


Our data provide further evidence that remote instruction exacerbated existing inequities 
by severely and disproportionately limiting the learning opportunities for students from low- 
income and Black communities. Teachers estimated that, on average, only 60% of students were 
engaged in remote learning on a regular basis, with wide gaps in perceived engagement along 
racial and socioeconomic lines. Teachers working in high-poverty schools and in schools that 
serve a majority of Black students reported that their students were less likely to have the 
technology required to access online learning resources and, consequently, less likely to 
regularly engage in remote learning activities. 

Not surprisingly, teachers experienced a precipitous drop in their self-reported sense of 
success during the pandemic relative to both fall baseline levels and retrospective reports from 
the spring before schools closed. We find that 53% of teachers reported a decline in their sense 
of success based on our retrospective survey data. Simultaneously, schools’ efforts to support 
teachers during the crisis did appear to matter. Our exploratory analyses suggest that schools 
with more supportive remote working conditions were more successful at helping their teachers 
maintain a sense of success during the pandemic. Specifically, we find that teachers were less 
likely to experience declines in their sense of success when they worked in schools that 
communicated effectively, provided targeted professional development, recognized teachers’ 
efforts, facilitated meaningful collaboration, and held fair expectations during the pandemic. 

Our findings directly inform ongoing efforts to deliver high-quality educational 
opportunities to all students as the pandemic persists. They also contribute to the broader 
literature on teachers’ working conditions and educational reform. We illustrate for the first time 
the large variability in teachers’ self-reported working conditions within schools and over time. 


School work environments are not a monolith. Rather, they are shaped by a dynamic set of 


organizational practices that can change over the course of the academic year and are 
experienced in different ways by teachers in the same school. We also highlight the importance 
of working conditions as a potential moderator in education interventions that necessitate 
organizational change. Schools with stronger working conditions are likely to be much more 
successful at supporting teachers to navigate organizational change during crisis and in future 


school change efforts. 


Prior Literature 

The Importance of Working Conditions in Schools 

Over the last decade, studies capitalizing on new measures of work environments 
constructed from student, teacher, and principal responses to district and statewide surveys have 
demonstrated how professional contexts influence teachers’ work. Researchers have consistently 
found that working conditions are strongly associated with teacher satisfaction and student 
academic growth. For example, Johnson, Kraft, and Papay (2012) analyzed data from the 
statewide Massachusetts Teaching Learning and Leading (MassTeLLS) survey and concluded 
that measures of school professional context explained a large portion of the apparent 
relationship between student demographic characteristics and both teacher satisfaction and career 
intentions. They also found that favorable working conditions predicted higher rates of academic 
growth, as measured by standardized tests, even in schools serving similar student populations. 
The working conditions that mattered most to teachers included the principal’s leadership, 
collegial relationships, and the school’s organizational culture. 

Studies have repeatedly demonstrated that this combination of social working conditions 


influences teachers’ satisfaction and ultimately affects their decisions about whether to stay or 


leave their schools (see Simon & Johnson, 2015 for a review). For example, Ladd (2011) found 
that the quality of school leadership and planning time with colleagues predict school 
effectiveness, as measured by student achievement. Likewise, Jackson and Bruegmann (2009) 
found that students perform better on standardized tests when their teachers work in a school 
with more effective colleagues. Most recently, Viano and colleagues (2020) used an adaptive 
conjoint analysis survey design to show how teachers’ career decisions are strongly influenced 
by consistent administrative support and enforcement of discipline—two “malleable processes” 
(p. 1) within administrators’ locus of control. 

Working Conditions’ Influence on Teachers’ Sense of Success. Teachers’ working 
conditions have a profound influence on teachers’ sense of success with students. In their 
seminal study of 50 new teachers in Massachusetts, Johnson and Birkeland (2003) found that, 
when making decisions about whether to stay in their schools, transfer to a different school, or 
leave the profession, the most important factor for teachers was whether they believed they could 
be effective with their students. Teachers described the myriad ways in which social working 
conditions either supported or thwarted their sense of success. 

Decades of research has found that teachers’ own perceptions of their effectiveness to be 
a factor of important consequence for students. Grounded in Rotter’s (1966) locus of control 
theory and Bandura’s (1977) social cognitive theory, a range of teacher efficacy constructs have 
been linked with student outcomes. Tshannen-Moran and colleagues (1998) review the literature 
on teacher self-efficacy and find evidence of strong positive associations between teacher self- 
efficacy and student achievement, motivation, and students’ own sense of self-efficacy. Critical 
for our context, they find that teachers with greater self-efficacy are more persistent and resilient 


when faced with challenges and setbacks. At the school level, they find that teachers’ efficacy is 


a strong predictor of overall organizational climate. Others have since demonstrated the positive 
link between collective efficacy and a range of student outcomes (Goddard et al., 2004; 
Moolenaar et al., 2012). 

Working Conditions and Organizational Change. The extent to which a school is 
supportive of teachers and well-functioning as an organization can greatly influence how 
effectively a school pivots during times of change. School leadership is especially critical during 
transitions because—as Bryk and colleagues reported—the most basic responsibility of 
principals is to ensure that schools “work properly” (Bryk et al., 2010, p. 62). Grissom and Loeb 
(2011) identified principals’ organizational management skills as a predictor of student 
achievement and suggested that, without attention to organizational management, principals 
rarely produced overall school improvement. Likewise, Reinhorn, Johnson, and Simon (2017) 
found that principals of successful schools deliberately craft a professional culture that promotes 
continuous improvement for teachers. 

Studies of large-scale school reforms provide insight into the power of working 
conditions in school improvement efforts. In their longitudinal investigation of Chicago 
elementary schools over fifteen years, Bryk and colleagues (2010) developed a framework of 
five “essential supports” that are correlated with school improvement: (1) a coherent 
instructional guidance system; (2) the school’s professional capacity; (3) strong parent- 
community school ties; (4) a student-centered learning climate; and (5) leadership that drives 
change. Using survey data from principals, teachers, and students, the researchers found that 
schools with strong essential supports were ten times more likely to improve student 
performance on standardized tests than schools with weak supports. 


The Challenges of Teaching During Crisis 


Organizational change in schools is not always well-developed and carefully planned in 
advance. When external crisis strikes, schools must shift their practices suddenly—often in ways 
that are impossible to anticipate. Teachers are “first responders in tragedy” (O’Toole & Friesen, 
p. 1), and schools are integral to both the first-line and long-term post-disaster response (L. 
Evans & Oehler-Stinnett, 2006; Wolmer et al., 2003). Studies of teachers following disasters 
demonstrate that teachers frequently endure the stress of supporting students’ socioemotional and 
academic needs while tending to their personal trauma (Carlson et al., 2010; Kuriansky, 2013). 
Following Hurricane Katrina in New Orleans, for example, Lowery and Burts (2007) found that 
teachers described teaching under “survival circumstances” (p. 72)—coping with their own 
material losses and elevated rates of depression and fear while supporting students. 

Crisis often disproportionately affects low-income communities and the schools that 
serve them (Evans, 2004; Simon & Evans, 2014). In ordinary times, Bryk (2010) described 
schools serving large proportions of children living under “extraordinary circumstances— 
neglect, abuse, homelessness, foster care, domestic violence” (p. 29) as facing a “three-strike” 
problem: they are highly-stressed organizations, existing in challenged communities, and 
confronting tremendous human need every day. Supporting students and their families in coping 
with additional stressors can exacerbate an already overwhelming teaching role (Bryk, 2010; 
Kraft, Papay, Johnson, Charner-Laird, Ng, & Reinhorn, 2015; Simon & Evans, 2014). As such, 
during crises, teachers frequently experience “role overload” (Kuntz et al., 2013) and a decreased 
sense of self-efficacy in their teaching (Seyle et al., 2013). 

Teachers’ Experiences with “Emergency Remote Teaching” During COVID-19 
In March 2020, the COVID-19 virus sparked widespread, sustained disruption to 


schooling, shuttering school buildings and forcing instruction online within a matter of days. 


Early reports on the transition to “emergency remote teaching” (Hodges et al., 2020) by Gallup 
and others suggest that the overwhelming majority of schools pivoted quickly to some form of 
remote instruction (Brenan, 2020; Hamilton et al., 2020). In a nationally representative survey of 
K-12 public school teachers and principals, a study by researchers at RAND found that by May, 
99% of teachers reported facilitating remote instruction (Hamilton et al., 2020). For most public 
school teachers, this entailed mastering an entirely new set of pedagogical practices for teaching 
online (Trust & Whalen, 2020). 

Instructional Approaches & Academic Rigor. Across school systems, the approach to and 
rigor of distance learning varied widely. Researchers at the Center on Reinventing Public 
Education found that just one-third of districts held teachers accountable for providing 
instruction, tracking student engagement, or monitoring students’ academic progress (Gross and 
Opalka, 2020). Similarly, a nationally representative analysis of school websites by the 
Education Research Alliance for New Orleans found considerable variation in the degree of 
synchronous instruction, progress monitoring, equity of access and breadth of service provided 
by schools. They also found that the educational attainment of parents and other neighborhood 
adults was the strongest predictor of the extent of educational activities schools provided (Harris 
et al., 2020). Nearly two-thirds of teachers (64%) surveyed by EdWeek reported that there were 
“no consequences” for students who failed to meet standards during the coronavirus closures. 

The Experiences of Educators. Multiple studies explored the experiences of educators as 
they implemented emergency remote instruction. Researchers at RAND found that nearly 90% 
of principals reported that students in their schools lacked internet access and 40% reported that, 
for teachers, access to technology and/or internet was a barrier. Roughly a third of principals 


(35%) reported that district policies related to the use of online tools (such as Zoom or Google 


Suite) presented limitations (Hamilton et al., 2020). In addition, EdWeek (2020) found that 74% 
of teachers said their students current level of engagement was “much lower” or “somewhat 
lower” than it had been prior to the pandemic, and nearly a quarter of students were “essentially 
‘truant.’” Finally, the EdWeek survey also showed that morale for teachers, students, and 
administrators across the country plummeted during the early months of the pandemic. 
Interviews with 40 teachers across the country about their daily experiences found that teachers 
expressed concerns on three key themes: struggling to motivate students virtually; the loss of 
professional identity and burnout; and exacerbated inequities for students (Reich et al., 2020). 
Exacerbated inequalities. Several studies have highlighted the stark disparities in student 
engagement between schools serving low-income students and students of color and those 
serving wealthier, whiter communities (Hamilton et al., 2020; Patrick & Newsome, 2020). For 
example, Educators for Excellence (2020) found that just 51% of teachers in high-poverty 
schools reported that most of their students participated daily in distance learning, in comparison 
with 84% of teachers in affluent schools. Bacher-Hicks, Goodman and Mulhern (2020) found 
similar patterns in their study of high frequency internet search data: in more populous regions 
with greater wealth and superior internet, more households sought out online learning resources. 
Expectations for students living in poverty were also lower. A national survey of school 
leaders by researchers at American Institute for Research (2020) revealed that students in high- 
poverty districts were expected to spend far less time on instructional activities than were their 
peers in low-poverty districts. Teachers in such districts were also far less likely to hold 
synchronous classes or to provide live virtual instructional support. Further, 29% of leaders in 
high-poverty districts said that distance learning for elementary school students primarily 


involved content review rather than teaching new material compared to just 8% in low-poverty 
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districts. Further, many administrators who responded to open-ended items on the AIR survey 
voiced concerns about students with disabilities, English Language Learners, and children living 
in poverty. Researchers have expressed similar concerns that pandemic-induced closures will be 
most severe for low-income students (Bacher-Hicks et al., 2020; Chetty et al., 2020; Kuhfeld et 
al., 2020). 

Little research has considered the role of working conditions in the transition to remote 
learning during the COVID-19 pandemic. Greenberg (2020) surveyed a small sample of schools 
supported by the Silicon Schools Fund, a California foundation led by the author, and found that 
the schools that experienced the greatest success with the implementation of emergency remote 
teaching were able to leverage their flexible, team-oriented cultures as they learned a new way to 
deliver instruction. We build on both the established literature on working conditions and the 
emerging body of research exploring teachers’ experiences during the COVID-19 pandemic to 


explore how working conditions matter during times of organizational change. 


Data & Methods 

Data 
The Sample 

We draw on teacher surveys from the fall and spring of the 2019-2020 school year with a 
sampling frame that included all teachers in 206 schools across 15 districts throughout the 
southern, midwestern and eastern United States. Much of our analyses describe responses from 
the spring survey, which was administered by Upbeat in coordination with districts between 
April 27th and June 23rd, 2020. Recruitment emails emphasized the goal of amplifying teachers’ 


perspectives about their working conditions and the confidential nature of the third-party survey. 
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A diverse sample of 7,841 teachers working across nine geographically-diverse states (GA, IL, 
LA, MI, NY, SC, TX, VA, VT) responded to the survey with an overall response rate of 81%. 
We combined these data with responses from a prior survey administered to the same schools in 
the fall before the pandemic, between September 3rd, 2019 and January 31st, 2020 (77% 
response rate). A total of 5,957 teachers responded to both the fall and spring survey waves, 
which we link using teachers’ unique email addresses. Together, these surveys provide 
information on teacher self-reports of their own sociodemographic characteristics, as well as 
their experiences and perceptions before and during the COVID-19 pandemic. We compliment 
these survey data with school sociodemographic characteristics gathered from state departments 
of education and school websites. 

As shown in Table 1, the respondents represent a diverse sample of teachers in traditional 
and charter public schools working across a wide range of student populations. Roughly 80% of 
teachers are female and average about 13 years of experience, closely approximating national 
averages (Taie & Goldring, 2020). Similarly, White, non-Hispanic teachers comprise three- 
quarters of the sample and 79% of the nation’s public school teachers. Black teachers are 
oversampled relative to the national workforce, comprising 14% of the sample, but just 7% of 
the nation’s teachers. Conversely, only 3% of our sample is Hispanic compared to 9% of the 
nation’s teachers. Their schools served student populations that were, on average, 30% Black, 
18% Hispanic, and 45% White, 54% low-income families and 14% students with disabilities. 

We collected a range of data about the contexts and transition to remote learning for 
districts and charter management organizations in the sample (see Appendix Table A1). Districts 
transitioned to remote learning between March 13th, 2020 and April 13th, 2020. COVID-19 case 


counts were slightly lower in our sample, on average, than the nation as a whole, but infection 
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rates were rising more rapidly relative to other parts of the country.’ At the start of remote 
learning, students in many districts worked on instructional packets distributed on the last day of 
in-person instruction. As state governors extended school closures, districts began working on 
long-term instructional methods. Most districts depended on Google Classroom and Google 
Meet or Zoom. 

All of the districts we study took steps to ensure students could engage in remote learning 
including distributing devices and printed work packets to students. Some districts also deployed 
WiFi-enabled buses to neighborhoods or provided WiFi in school parking lots. Expectations of 
both teachers’ and students’ work varied across and within districts. Some districts required 
synchronous instruction, while others did not. In our sample, charter school networks typically 
required more face-to-face, remote instruction than did public schools. Some districts required 
teachers to provide weekly student feedback or host weekly office hours. All districts altered 
their grading policies, frequently in response to state guidance. In some districts, grading stopped 
altogether after closure, and final grades reflected pre-COVID-19 achievements. 

Instruments and Measures 

We draw on the Upbeat Teacher Engagement survey that we designed in partnership with 
Upbeat to capture teachers’ perceptions about their work environments. The survey is centered 
on a range of constructs identified in the research literature as relevant for teachers’ satisfaction, 
sense of success, and retention (Kraft, 2017). District and schools hire Upbeat to conduct the 
survey on their behalf. We augmented the Teacher Engagement spring survey with the Teaching 


from Home survey, which consisted of a parsimonious set of 14 items we designed to capture 


1 The COVID-19 county case count in our sample was 6.56 for every 100,000 residents on April 20th, slightly lower 
than the national average of 7.73. However, the Rt scores in the sample, a measure of how fast the virus was 
spreading, was 1.81, meaningfully higher than the national average of 1.22. 
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teachers’ experiences and perceptions about their working conditions during emergency remote 
teaching. We developed these items by gathering practitioner feedback from Upbeat client 
districts, conducting cognitive testing and interviews, and then pilot testing the items (Dillman, 
Smyth, & Christian, 2009). 

The spring survey combined both the Teacher Engagement survey and Teaching from 
Home survey. Teachers were prompted to answer the Teacher Engagement survey based on 
“experiences working at your school PRIOR to COVID-19 school closures.” After completing 
these items, teachers were asked to respond to a set of questions about their “experience teaching 
remotely while schools are closed due to COVID-19.” We included the clause “during this 
distance learning period” at the end of each item on the Teaching from Home survey to ensure 
teachers responded based on their recent experiences during the pandemic. The vast majority of 
items on the Teacher Engagement and Teaching from Home surveys asked teachers to use a 
four-choice Likert response scale to express their agreement or disagreement with a statement 
(Strongly Disagree, Disagree, Agree, Strongly Agree). We provide further details about the 
survey instructions and items in Appendix A. 

Here, we highlight several items on the Teaching from Home survey that are central to 
our analyses. First, to understand challenges during remote learning, we included a series of 
questions about student access to technology and teachers’ success in balancing responsibilities 
at work and at home, caretaking responsibilities, and comfort with technological tools for remote 
teaching. We also asked teachers to report on the percent of students regularly engaged during 
remote learning. The answer choices for this question were in quintiles and then coded to 
indicate the median value of each quintile (e.g., if a teacher reported that 41-60% of their 


students were engaged, this was coded as 50.5%). Second, to examine sense of success and how 
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it changed over time, we asked teachers if they felt successful teaching students in the three time 

periods discussed above: the beginning of the school year (fall Teacher Engagement survey), the 

spring prior to remote learning (spring retrospective Teacher Engagement survey), and the spring 
during remote learning (spring Teaching from Home survey). 

Finally, to capture key elements of teachers’ working conditions during and prior to the 
pandemic, we asked teachers to report on between seven and nine items related to five broad 
constructs at each period in time: professional development, communication, recognition, 
collaboration, and professional expectations.? We used these survey responses to generate three 
composite measures of working conditions through a principal component analysis (PCA) at the 
teacher-level in order to reduce the dimensionality of the data. Similar to prior analyses of 
teacher working conditions surveys (e.g., Kraft, Marinell, & Yee, 2016; Kraft & Papay, 2014), 
each of the first principal components from the three analyses weighted each item roughly 
equally, explained half of the total variance,’ was the only component above the "breaking 
point," and was the only component with an eigenvalue greater than one. The items that 
contribute to each composite measure of working conditions have an alpha reliability of between 
0.86 and 0.89 in each survey period. 

We used the first principal component from our teacher-level PCA as a measure of 
teachers’ individual perceptions of their own working conditions. Such an approach has been 
infeasible in most prior research because working conditions surveys are often anonymous, 
requiring aggregation at the school level. A variance decomposition shows that only 15% of the 


total variation in these individual working condition measures captured at three points in time 


2 As shown in Appendix Table A2, the five constructs remained constant across periods with some variation in 
question word choice. 

3 The first principal component explained 51% in the variance among these survey items in the fall, 53% in the 
spring before remote learning, and 55% in the spring after remote learning. 
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reflects stable differences across schools. Almost half (47%) of the variation is across teachers at 
the same school, demonstrating that teachers in the same school experience their working 
conditions in very different ways. Over a third (38%) is within individual teachers over time 
revealing that working conditions can vary meaningfully over the course of a single school year. 
This is consistent with findings from prior qualitative studies that have demonstrated substantial 
nuance and complexity in how teachers experience their working conditions (Forman et al., 
2017; Griffin, 2018; Griffin & Tackie, 2016; Johnson, 2019; Kardos et al., 2001). 

While teacher-specific measures of their working conditions are attractive from a 
conceptual standpoint, they introduce potential bias when examining self-reported outcomes. 
Teachers' own perceptions of their working conditions may be endogenously related to their 
sense of success due to, for example, teachers’ overall satisfaction with their job. To guard 
against this threat, we also generated a “jackknife” working conditions measure that averages 
teacher-level scores of all teachers in a school but leaves out individual teachers’ own 
perceptions. This peer-average measure serves to characterize the overall perception of working 
conditions and break the link between individual teachers’ own perceptions of their working 
conditions and their self-reported sense of success. At the same time, this average measure fails 
to allow for any real differences in teachers’ experience within the same school. We view our 
individual working conditions and peer-average working conditions measures as providing 
approximate upper and lower bounds of the relationships we explore. We standardized our 
individual and peer-average working conditions measures to have a mean of zero and variance of 
one in each survey period. 


Methodology 
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To address our first two descriptive research questions, we report teachers’ responses to 
the spring Teaching from Home survey overall and across a range of sociodemographic 
characteristics of teachers and schools. For ease of interpretation, we use a binary coding scheme 
to describe if a teacher agreed or strongly agreed to an item. 

To explore how the COVID-19 pandemic and transition to remote learning affected 
teachers’ sense of success, we adopt a longitudinal growth model (Singer & Willett, 2003) that 
combines both pre-post and retrospective survey design features. From a causal inference 
perspective, our growth model can be thought of as an interrupted time series (Shadish, Cook, & 
Campell, 2002). The core assumption of this approach is that teachers’ sense of success would 
have remained relatively stable in the absence of the pandemic and that no other concurrent 
events affected teachers’ sense of success. We examine both assumptions directly in our 
robustness section. Throughout these analyses we conceptualize “treatment” as the joint effects 
of the sudden transition to remote teaching and working coupled with the broader health, 
economic, and social effects of the COVID-19 pandemic during which this transition took place. 
Thus, our findings identify the effect of remote teaching combined with the broader challenges 
teachers faced in their professional and personal lives due to the pandemic. 

Our longitudinal growth model captures how individual teachers’ sense of success 
changed across three time periods: fall 2019, spring 2020 before remote teaching, and spring 
2020 during remote teaching. This combined pre-post and retrospective study design provides 
two main benefits to the growth model analyses. First, the retrospective survey design captures 
teachers’ perceptions during the time period just before remote learning, while avoiding potential 
response-shift bias in which an individual’s frame of reference changes across survey 


administrations (Aucejo et al., 2020; Pratt, MacGuigan, & Katzev, 2000; Weixler, Harris & 
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Barrett, 2018). At the same time, there exists the concern that a retrospective survey design may 
lead to a halo effect in which teachers perceive the conditions during the pandemic as generally 
“bad,” biasing their responses to be consistent with schools being generally worse (Cooper, 
1981; Weixler, Harris & Barrett, 2018). The addition of the fall survey guards against this 
potential bias, as well as any recall bias associated with the challenges of remembering one’s 
experiences retrospectively. 

We specify a multilevel ordered logistic model within a longitudinal growth model 
framework to examine how teachers’ odds of rating their sense of success higher (e.g., from 
Disagree to Agree) changed over the three time periods in a teacher-period panel dataset. We 
model the full range of changes in teachers’ responses on the ordinal Likert scale because many 
of the changes we observe are between the two positive responses (Agree vs. Strongly Agree) 
given teachers’ generally high ratings of their sense of success. For ease of interpretation, we 


present the structural elements as a latent-variable model: 


Ye = a(t = Remote) + az(t = SpringPre) + 0X; + pZ,+ (uj + gr). C) 


Here, Y;; is an unobserved latent measure of teacher j’s sense of success at time t. The ordered 


logistic model takes the observed ordinal measure Y;, and estimates both the coefficients of the 


structural model as well as the cutpoints k, through k,,_, such that 


Yi =m if Km1S¥jip<Km form=1toD, (2) 
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where m is a given Likert scale response option and D=4. The estimate of ( is our primary 
parameter of interest, representing the log odds that a teacher rated their sense of success as 
higher during the pandemic relative to the fall, and X; and Z, are vectors of teacher- and school- 
level control variables, respectively. 4; captures teacher random effects and €;,; represents the 
idiosyncratic error term. Teacher-level controls include a series of indicator variables for teacher 
race, gender, and level of experience between early (less than five years), mid (between five and 
15 years), and late (over 15 years) career teachers.* School-level control variables include student 
enrollment, student racial/ethnic demographics, the percent of English language learners, and the 
percent of students eligible for free- or reduced-price lunch (FRPL). 

We next explore the role that teachers’ working conditions during the pandemic played in 
moderating changes in their sense of success. To do this, we interact individual (or peer-average) 
working conditions with our indicator for the remote teaching period, Remote. Our latent- 


variable model takes the form: 


Y; 


it = a(t = Remote) + B([t = Remote] * WG nemaie) + a,(t = SpringPre) + 0X; + 


pZ, + (uj + gr). (@) 


Here, our parameter of interest, 6, now represents the degree to which changes in individual 
teachers’ sense of success during the pandemic were related to the quality of their remote 


working conditions (WC). 


4 We address missingness for our teacher experience measure by creating and including an additional indicator for 
when teachers failed to report their own experience level. 
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We complement our longitudinal modeling approach with a taxonomy of ordered logistic 
regression models within a value-added framework. These value-added models (VAMs) allow us 
to estimate whether teachers’ sense of success during the pandemic is related to their remote 
working conditions, while controlling flexibly for teachers’ self-reported sense of success in two 
prior periods and a range of teacher and school characteristics. In this analysis, the structural 


component of our latent-variable model takes the form: 


Vi aRenote = at Om * 1(¥}c=springPre =m) + oe Om * 1(¥}t=rau =m) + 


B(WC;+=remote) +e 0X; + OL. + ej: (4) 


Here, we include indicators for teachers’ sense of success from the spring retrospective survey 
pre-COVID-19 (SpringPre) and the fall survey (Fall) as non-parametric controls for teachers’ 
sense of success prior to the pandemic. Additional teacher and school-level controls are the same 
as described above. Across all our models we present results in terms of odds ratios and cluster 
our standard errors at the school level to account for the non-independence of teachers’ error 


terms within schools. 


Findings 


What challenges did the pandemic and sudden change to remote teaching present for 
teachers? 


Teachers’ responses to the Teaching from Home survey reveal that they faced a range of 
challenges in pivoting to remote teaching during the COVID-19 pandemic. As shown in Table 2, 
teachers perceived that, on average, only 59% of their students were able to regularly engage in 


remote learning. One factor that likely contributed to this low level of engagement is limited 
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access to reliable, high-speed internet and internet-enabled devices. Overall, 26% of teachers 
reported that their students lacked the technological tools they needed for remote learning. 

Teachers also reported concerns about their ability to manage their professional 
responsibilities and use online teaching tools. 40% agreed that caretaking responsibilities for 
children and/or dependent adults had made it difficult to do their job, and 16% of teachers 
reported that they were unable to balance their work with their other responsibilities at home. In 
addition, 8% of teachers were uncomfortable using the technological tools necessary to teach 
remotely. 

How did these challenges differ across school and teacher characteristics? 

School Characteristics. The low levels of perceived student engagement mask 
considerable variation across the schools in our sample. In Figure 1, we illustrate how average 
levels of perceived student engagement varied across schools. Roughly one in ten schools had 
less than 40% of students regularly engaged in remote learning, while approximately one in six 
schools had over 70% of students regularly engaged in remote learning. 

We also find stark differences in perceived student engagement along racial and 
socioeconomic lines. As illustrated in Figure 2, teachers in high-poverty schools report that only 
51% of students regularly engaged in remote learning, whereas their counterparts at low-poverty 
schools engaged at a much higher rate — 75%.° Figure 3 shows that in schools where a majority 
of students are Black, teachers reported that 47% of students regularly engaged in remote 


learning; at schools that enroll fewer than 10% of Black students, teachers reported that 71% of 


° Consistent with federal guidelines, we define low-poverty as schools where less than 25% of students are eligible 
for FRPL and high-poverty as schools where over 75% of students are eligible for FRPL. 
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students regularly engaged in remote learning. However, we find a much smaller difference in 
reported student engagement across schools based on the proportion of Hispanic students. 
Teacher Characteristics. Disaggregating overall patterns in teachers’ experiences reveals 
important differences in the challenges created by the pandemic and pivot to remote learning. 
Mid-career teachers particularly struggled to balance their work with the demands of their 
personal lives during the pandemic. In Figures 4 and 5, we show how challenges related to 
caretaking and balancing work and home responsibilities differed by experience and gender. Just 
over half (51%) of mid-career teachers reported that caretaking responsibilities made their job 
difficult, while 39% of early-career teachers and 35% late-career teachers reported these same 
concerns. These patterns likely reflect the greater likelihood that mid-career teachers have 
school-age children at home with them. Late-career teachers also report being less comfortable 
teaching online. As shown in Figure 6, 13% of teachers with 20-29 years of experience and 22% 
of teachers with 30 years of experience or more were not comfortable using online teaching 
tools, compared to only 6% of teachers with less than 10 years of experience and 7% of teachers 
with 10-19 years of experience. 
How did the pandemic and transition to remote teaching affect teachers’ sense of success? 
We find that the pandemic and the pivot to remote teaching resulted in a sudden and steep 
drop in teachers’ sense of success. As we show in Figure 7, the percentage of teachers who 
agreed or strongly agreed that they felt successful increased slightly from 92% in the fall to 96% 
in the spring prior to the pandemic, but then fell dramatically to 73% during the crisis. This drop 
in overall agreement rates actually understates the full decline in teachers’ sense of success 


given, for example, that it fails to capture those teachers who went from strongly agreeing to 


5 These results are quite similar even when we limit our sample to only include Black teachers. 
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only agreeing they felt successful. Overall, 53% of all teachers who completed the spring survey 
reported that their sense of success declined during remote teaching relative to just before the 
pandemic (see Figure 8). Of those reporting declines, roughly one in four reported a decline of 
more than one response level (e.g., moving from Strongly Agree to Disagree). 

We formally model the longitudinal changes to teachers’ sense of success using an 
ordered logistic regression model and report the results in Table 3. Interpreting our results from 
Table 3, we find the odds that teachers’ sense of success increased (e.g., Agree to Strongly 
Agree) dropped by 85% during the pandemic relative to the fall. Said in a more intuitive way, we 
find an almost 7-fold rise (or a 678% increase) in the odds that teachers’ sense of success 
decreased in the spring remote teaching period relative to the fall.’ We can also characterize this 
decline as a change in the continuous, latent measure of teachers’ sense of success. Our estimate 
suggest that teachers’ sense of success dropped by approximately 0.9 standard deviations relative 
to the fall.® 


What role did working conditions play in sustaining teachers’ sense of success during the 
transition to remote teaching? 


We find that teachers’ working conditions during the pandemic played an important role 
in sustaining their sense of success. We begin by providing graphical intuition for the role of 
remote work conditions in mitigating the decline in teachers’ sense of success. In Figure 9, we 
show how the percent of teachers who agreed or strongly agreed that they feel successful 
changed differentially based on the quintile of individual working conditions teachers reported 


during the pandemic. Although teachers whose schools struggled to provide strong remote 


? This estimate is derived by taking the inverse of odd ratios (see Long & Freese, 2014; p. 338) 
* This translation involves using sample estimates to approximate o,,, the standard deviation of the latent continuous 


measure of teachers’ sense of success and standardizing the untransformed log-odds estimate of 6 (Long & Freese, 
2014; p.333). 
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working conditions reported lower levels of success prior to the pandemic, they also experienced 
meaningfully larger declines in their sense of success on average. 

Growth Model Estimates. We formally model changes across the full spectrum of 
teachers’ sense of success using a longitudinal growth model. As shown in Table 3 columns 3 
and 4, we find that teachers with one standard deviation higher individual working conditions 
during the pandemic have more than twice the odds of their sense of success increasing during 
remote teaching compared to a teacher with remote working conditions one standard deviation 
lower. Alternatively, we can say that teachers working in schools with better remote working 
conditions were substantially less likely to experience a decline in their sense of success. We 
estimate that teachers at the 25" percentile of individual working conditions experienced a 10- 
fold increase in the odds that their sense of success decreased during remote learning, while 
teachers at the 75" percentile of individual working conditions experienced only a 4-fold 
increase. Our estimates are nearly identical with and without controls for teacher and school 
characteristics to the model. When we substitute individual working condition measures with 
peer-averages (columns 5 and 6), we find that this positive, statistically significant relationship 
persists but is attenuated: teachers in a school with one standard deviation higher peer-average 
working conditions have 15% greater odds that their sense of success increased compared to 
teachers with peer-average working conditions one standard deviation lower. 

VAM Estimates. Results from our value-added specification reported in Table 4 are quite 
similar to our findings from longitudinal growth models. We again find that the odds that 
teachers’ sense of success increased are more than double for a one standard deviation higher 
level of individual working conditions during the pandemic (Panel A). We also find that the odds 


of teachers’ sense of success increasing are roughly 30% greater for a one standard deviation 
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higher level of peer-average remote working conditions (Panel B). This is equivalent to a 0.4 
standard deviation increase in teachers’ sense of success for every standard deviation increase in 
individual working conditions and a 0.13 standard deviation increase in peer-average working 
conditions during the pandemic. Our estimates are quite robust to the inclusion or exclusion of a 
range of controls, including indicators for prior levels of reported success in both the fall and 
spring before remote learning, teacher and school characteristics, and prior ratings of teachers’ 
working conditions. 

We provide further intuition about the relationships between teachers’ individual working 
conditions and their sense of success during the pandemic in Figure 10. Here we plot the 
predicted probabilities of positive or negative responses to the sense of success item, conditional 
on the full set of indicators for prior sense of success and other controls (Table 4 column 4). 
These conditional probabilities illustrate the strong association between individual working 
conditions and teachers’ sense of success during the pandemic even after controlling for their 
prior levels of self-reported success and other covariates. We see the probability of feeling 
successful (Agree or Strongly Agree) during remote learning increases from 25% at the low end 
of the remote working conditions distribution to 90% at the top end of the distribution. 
Extensions 

Our primary analyses focus on the relationship between teachers’ remote working 
conditions and changes in teachers’ sense of success. We extend these analyses by exploring 
how working conditions measured in the fall and spring prior to the pandemic relate to changes 
in teachers’ sense of success. We find some evidence that working conditions prior to the 
pandemic are related to changes in teachers’ sense of success, but to a much lesser degree than 


remote working conditions. Results from our growth model presented in Appendix Table A3 
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show largely positive (odds ratios greater than 1) but imprecise estimates for the interactions 
with fall and spring in-person working conditions. Results from VAM models shown in 
Appendix Table A4 are uniformly positive and at least marginally significant for these earlier 
measures of working conditions as predictors of teachers’ changes in their sense of success. This 
overall pattern of findings makes sense given that schools with stronger working conditions in 
the fall were more likely, but not guaranteed, to have stronger working conditions in the spring 
during the pandemic (individual r=0.48; peer-average 7=0.68; see Appendix Table A5). 
Robustness Tests 

We examine the robustness of the descriptive patterns and effects on teachers’ sense of 
success in several ways. First, our findings, particularly the differential levels of student 
engagement across schools and the sudden drop in teachers’ sense of success, could reflect the 
influence of George Floyd’s murder and the subsequent international uprising in response to 
racial injustice and police violence. However, 91% teachers responded to the spring Teacher 
Engagement and Teaching from Home surveys before May 25" when Floyd was killed. All of 
our findings remain quite consistent when we restrict our sample to include only those response 
before May 25". 

Second, attributing the large drop in teachers’ sense of success to the pandemic and 
transition to remote teaching assumes that their sense of success would have remained 
unchanged in the absence of this treatment. It is difficult to know for sure how teachers’ sense of 
success might have evolved in this counterfactual scenario. One approach would be to project the 
small upward trend in teachers’ sense of success between the fall and spring prior to the 
pandemic. If this trend were sustained, as we might expect given the improvement of teachers’ 


experience on the job (Kraft & Papay, 2014), our estimates would understate the full effect of the 
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pandemic and remote teaching on teachers’ sense of success. Alternatively, we might assume 
teachers’ sense of success would fall as the end of the school year neared and they experienced 
burnout (Parker et al., 2012). We specify our growth model such that our estimate of the decline 
in teachers’ sense of success is relative to the fall, implicitly assuming a counterfactual decline in 
their sense of success back to their baseline levels (possibly because of burnout). Ultimately, 
end-of-year burnout for teachers would need to be implausibly large, multiple orders of 
magnitude larger than the gains teachers made from fall to spring, to even come close to calling 
into question the conclusion that the pandemic had a large negative causal effect on teachers’ 
sense of success. 

Finally, one mid-sized suburban district in the south constitutes 61% of our sample and 
102 of 206 total schools (see Appendix Table A1). It is possible this district is driving the overall 
patterns we find. However, when we remove this district from the sample our results remain 
broadly similar. If anything, reported challenges appear even more daunting with this mid-sized 
suburban district removed from our sample. For example, in a restricted sample, reported student 
engagement is notably lower (52% vs. 64% in removed district), perceived challenges with 
student use of technology are greater (32% vs. 22%) and teacher discomfort with technology is 
higher (10% vs. 8%). Despite reporting more challenges, slightly fewer teachers in the restricted 
sample reported a decline in their sense of success (50%) compared to the mid-sized suburban 


district (54%). 


Discussion 
In this paper, we examine the experiences of teachers during the early months of the 


COVID-19 pandemic—a period of unprecedented and remarkably rapid organizational change. 
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Our findings reveal that teachers across the U.S. faced immense challenges during this time. The 
pivot to emergency remote teaching resulted in a sudden, massive drop in teachers’ sense of 
success. Teachers struggled to find a balance between their professional and personal 
responsibilities. They scrambled to master new technology. And, often despite dedicated efforts 
by districts and schools to distribute technology and provide internet access, teachers reported 
that large proportions of their students remained disengaged in remote learning—due, in part, to 
their continued lack of access to technology. 

Although teachers in every career phase, life stage, geographic region, and school-type 
were challenged by the transition to emergency remote teaching, hardships differed in type and 
in magnitude. Consistent with prior research, however, our findings highlight the critical 
importance of school organizational practices to teachers’ work. A schools’ working conditions 
during the pandemic mattered greatly for sustaining teachers’ sense of success. We find that 
teachers who could depend on strong communication, fair expectations, and a recognition of 
effort from the top, along with targeted professional development and facilitated, meaningful 
collaboration with colleagues, were least likely to experience a dip in their sense of success. 

Teachers working in low-income communities and communities of color that have been 
disproportionately affected by the virus have faced the most profound challenges (Oppel Jr. et 
al., 2020). Early evidence suggests that the pandemic has already exacerbated existing 
educational inequality (Bacher-Hicks et al., 2020; Chetty et al., 2020; Kuhfeld et al., 2020). We 
provide further evidence that learning was deeply obstructed for Black students and for those 
with limited family income. Teachers estimated that, on average, only half of students in high- 
poverty and/or majority Black schools regularly engaged in remote learning. In addition, our data 


shed light on the more subtle ways in which students are shortchanged: by attending schools with 
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dysfunctional work environments that are unable to support teachers to adapt their practice 
during challenging times of sudden change. When teachers cannot do their work, students 
ultimately suffer (Kraft, Marinell, & Yee, 2016). 
Implications for Policy & Practice 

Policymakers and practitioners can take concrete action to ensure that, as the pandemic 
persists, schools are better set up to support teachers’ work, and in turn, students’ learning. The 
shift to emergency remote teaching was sudden and messy, but catastrophe can be a catalyst for 
positive change. Building and sustaining strong work environments for teachers—whether 
remote or in-person—should be central to the approach in every locale. However, strong 
working conditions will not develop in response to state or district mandates. To ensure that 
policies translate into teaching and learning, policymakers must focus on capacity-building 
(McDonnell & Elmore, 1987) in schools, in districts, and in state education offices intended to 
support the schools’ work. 

This study furthers our understanding of where capacity-building efforts might focus. 
First, principals need support in solving what Heifetz and colleagues (2009) call “technical 
problems”—those challenges that “have known solutions that can be implemented by current 
know-how” (p. 19). For example, equipping students with the necessary technology and internet 
access or ensuring childcare for teachers’ children. Without solutions to these hard, but solvable, 
problems, principals and teachers simply cannot do their jobs. Easing the process of addressing 
technical problems can allow principals to focus their limited time and energy in solving 
“adaptive challenges”—those that require leaders to “build new ways of being and responding 


beyond the current repertoires of available know-how.” (p. 2) Responding to such challenges 
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depends on courageous leadership, as it requires improvisation, experimentation, and 
engagement of stakeholders affected by the work. 

Once schools can focus on addressing adaptive challenges, principals can concentrate on 
strengthening the working conditions that influence teachers’ work and affect a school’s ability 
to engage in organizational change. To start, principals can narrow their efforts by defining their 
school’s “malleable processes” (Viano et al., 2020, p.1)—those within their locus of control. 
Encouragingly, our results suggest that what schools do during the pandemic to support teachers’ 
matters most. Schools might begin by addressing the processes that our study highlights as 
important to teachers in times of change, such as developing systems for strong communication 
and for recognizing teachers’ efforts. In addition, they might work with teachers to solicit and set 
expectations for work, determine training that teachers need, and design structures for formal and 
informal collaboration. By investing in these aspects of working conditions, schools will be 
better prepared to navigate organizational change. Weathering such change—especially during 
an ongoing and unpredictable crisis—depends on a culture where both organizational systems 
and interpersonal relationships exist and can support the improvisation and swift action required. 

As part of this work, schools need support to address those problems that influence the 
work of teachers and students but fall outside of the school’s locus of control. For example, as 
our results indicate, teachers in schools with mostly Black students or students living in poverty 
reported much lower rates of student engagement in remote learning. Although access to 
technology and internet likely contribute to this finding, policymakers, practitioners, and 
researchers must not overlook the intersectionality of economic disadvantage, racism, and other 
powerful out-of-school factors that constrain teachers’ work and students’ opportunities to learn 


(Brody et al., 2006; Duncan & Brooks-Gunn, 1997; Mays et al., 2007; Seaton & Yip, 2009). 
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Conclusion 

Our study demonstrates that now—more than ever—school organizational practices 
matter. Teachers overwhelmingly enter the profession because they want to make a difference in 
the lives of students—especially for those who have been long underserved (Kraft et al., 2015). 
Yet, too often teachers find themselves in contexts where they feel isolated and their working 
conditions impede their teaching. When this happens—and teachers cannot achieve a sense of 
success—they leave (Johnson & Birkland, 2003). We build on this work by documenting the 
enormous challenges that teachers faced during the onset of the COVID-19 pandemic, a time of 
unprecedented and rapid organizational change. We find that teachers experienced a precipitous 
drop in their sense of success during this sudden pivot to emergency remote teaching—and as 
their most vulnerable students struggled to engage in remote learning. Yet, supportive working 
conditions during remote learning played a protective role: in schools where teachers 
experienced stronger working conditions, they also reported much lower declines in their sense 
of success than their peers in schools with weaker working conditions. 

The pandemic has sparked enormous transformations for schools and teachers’ practices. 
To weather this ongoing storm, schools need to provide the supports that educators require to 
succeed in this new, evolving context. Teachers do not work in a vacuum. Efforts to enact both 
temporary and lasting change in schools must attend to the organizational conditions that make 


teachers’ work and students’ learning possible. 
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Tables 
Table 1. Sample Characteristics 


Average au ‘ pu : 
Percentile Percentile 

Panel A: Teachers 
Experience 13.1 5.0 19.0 
Female 0.81 
Asian 0.01 
Black 0.14 
Hispanic/Latinx 0.03 
White 0.75 
Multi-racial 0.01 
n 7,841 

Panel B: Schools 
Asian Students 0.01 0.00 0.01 
Black Students 0.30 0.14 0.36 
Hispanic/Latinx Students 0.18 0.07 0.19 
White Students 0.45 0.16 0.64 
Multiracial Students 0.02 0.00 0.03 
Low-Income Students 0.54 0.37 0.67 
Students with Disabilities 0.14 0.14 0.15 
Elementary Schools 0.47 
Middle Schools 0.20 
High Schools 0.22 
Mixed Grades School 0.11 
n 206 

Panel C: Districts 
Student Enrollment 50,942 9,797 75,577 
State Rt on Announcement of School Closure 1.81 1.71 2.08 
New Cases per 100,000 Residents 6.56 4.10 4.10 
Urban 0.07 
Suburban 0.84 
Rural 0.10 
Traditional School District 0.94 
n 15 


Notes: School and district averages are weighted by the number of teachers in the sample. 
Teacher characteristics are based on teacher reports on the Teaching from Home survey. 
School characteristics are culled from the Common Core of Data, State Departments of 
Education, and individual school websites. Low-Income Students refers to students receiving 
free or reduced price lunch. District characteristics are based on the Common Core of Data 
triangulated with district websites and rt.live. See Table Al in the appendix for further 
information on school districts. 
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Table 2. Teachers’ Perceptions of the Challenges with Remote Teaching During the Pandemic by Teacher and School 


Characteristics 
Panel A. Teacher Characteristics 
Early Mid- Late 

Average Male Female Career Career Career Asian Black Hispanic White 
Percent of Students Engaged 59.4 57.5 60.0 59.3 62.4 57.8 55.5 51.3 61.3 61.4 
Lack of Student Access to Technology 26.4 21.7 26.9 27.0 22.4 28.3 24.0 26.6 28.4 25.2 
Balancing Work & Home 
Responsibilities 15.6 11.7 16.4 15.1 19.1 13.9 12.0 15.2 22.6 15.2 
Caretaking Responsibility 40.0 42.5 39.5 38.4 50.3 34.9 34.7 34.1 46.7 40.9 
Discomfort with Technological Tools 8.8 74 9.2 6.9 6.0 10.8 5.3 8.8 10.3 8.5 
n 7,841 1,380 6,358 1,162 2,319 4,363 75 1,073 261 5,898 

Panel B. School Characteristics 
<10% Majority <10% Majority Low High : ‘ 

a Black Black Hispanic Hispanic Poverty Poverty Elemeoiay, «Muda High 
Percent of Students Engaged 59.4 71.1 47.1 61.5 60.0 74.5 51.2 60.4 56.0 59.4 
Lack of Student Access to Technology 26.4 19.4 34.4 23.4 33.3 12.2 36.7 29.8 20.5 24.5 
Balancing Work & Home 
Responsibilities 15.6 19.6 14.1 16.0 14.4 16.5 14.1 14.7 16.6 15.7 
Caretaking Responsibility 40.0 41.3 37.5 40.9 40.1 43.3 36.8 40.3 39.6 41.3 
Discomfort with Technological Tools 8.8 10.5 7.0 8.7 7.7 8.2 8.6 9.8 6.3 78 
n 7,841 1,088 1,326 4,100 699 971 1,645 3,659 1,563 1,742 


Notes: We display the overall and subgroup averages for student engagement and other reported challenges. The Percent of Students Engaged is the 
average of teacher reports of the percent of students regularly engaged during remote learning. Answer choices for this question were in quintiles and 
then coded to indicate the median of each quintile. Other reported challenges are based on dichotomous variables indicating if teachers reported each 
construct as a challenge. Teacher experience was divided into three subgroups based on the years of experience: early career (less than five years), mid- 
career (between 5 and 15 years), and late-career (over 15 years). Consistent with federal guidelines, we define low-poverty as schools where less than 
25% of students are eligible for free- or reduced-price lunch (FRPL) and high-poverty as schools where over 75% of students are eligible for FRPL. 


Table 3. Longitudinal Growth Analysis of Teachers’ Sense of Success Moderated by Remote Working Conditions 


d) (2) (3) (4) (5) (6) 

Remote Learning O.148*** — O.147*** — Q.150*** = O.150*** — 0. 146*** = 0. 146*** 

[-30.16] [-30.28] [-28.89] [-28.97] [-30.30] [-30.38] 
Spring, in Person L.717*** — LL717***® 1 686*** = L.685*** L714 1714 

[13.10] [13.11] [13.03] [13.03] [13.10] [13.10] 
Remote Learning* Individual Working Conditions 2.069*** = 2.071*** 

[15.31] [15.49] 
Remote Learning* Peer-Average Working Conditions 1.168*** 1.146*** 
[3.11] [2.90] 

n(Observations) 17,871 17,871 17,871 17,871 17,871 17,871 
n(Teachers) 5,957 5,957 5,957 5,957 5,957 5,957 
Teacher and School Controls Yes Yes Yes 


Notes: * p<.05, ** p<0.01, *** p<0.001. Each column displays results from a separate ordered logistic regression of sense of success regressed on remote 
learning and, in panels 3-6, working conditions. Estimates are reported as proportional odds ratios. Robust standard errors are clustered at the school level 
and t-statistics are reported in brackets. Teachers’ sense of success is measured by ordinal questions on the Teacher Engagement and Teaching from Home 


surveys. Working conditions are measured by a PCA on a series of questions regarding professional development, communication, recognition, 


collaboration, and professional expectations from the Teaching From Home survey. Control variables include teacher race, gender, and level of experience, 
as well as school enrollment, student racial/ethnic demographics, the percent of English Language Learners, and the percent of students eligible for free or 


reduced priced lunch. 
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Table 4. Value-Added Models of the Relationship Between Remote Working Conditions and Changes in Teachers’ 
Sense of Success 


d) (2) (3) (4) 
Panel A: Individual Working Conditions 
Remote Working Conditions 2.72788 2.285% 2.250% ** 2.407 ** 
[22.99] [18.32] [18.44] [17.60] 


Panel B: Peer-Average Working Conditions 


Remote Working Conditions 1.310*** 1.180*** 1.153*** 1.305*** 
[6.15] [4.15] [3.96] [4.37] 
n 5,957 5,957 5,957 5,957 
Lagged Success Indicators Yes Yes Yes 
Teacher and School Controls Yes Yes 
Lagged Working Conditions Indicators Yes 


Notes: * p<.05, ** p<0.01, *** p<0.001. Each cell displays results from a separate ordered logistic regression of 
sense of success regressed on remote working conditions. Estimates are reported as proportional odds ratios. Robust 
standard errors are clustered at the school level and t-statistics are reported in brackets. Teacher sense of success is 
measured by ordinal questions on the Teacher Engagement and Teaching from Home surveys. Working conditions 
are measured by a PCA on a series of questions regarding professional development, communication, recognition, 
collaboration, and professional expectations from the Teacher Engagement and Teaching from Home surveys. See 
Table 3 for a list of control variables. 
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Figure 1. Variation in Average Student Engagement in Remote Learning Across Schools 


Notes: Means (solid line) and 95% confidence intervals (dashed lined) are reported for each school (n=206). 


The overall average (red line) across schools is 59%. 
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Figure 2. Differences in Student Engagement in Remote Learning by the Percent of Students 


from Low-Income Families in a School 
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Figure 3. Differences in Student Engagement in Remote Learning by the Percent of Black 
Students in a School 
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Figure 4. Challenges Balancing Work and Home Responsibilities by Teacher Experience and 
Gender 
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Figure 5. Challenges with Caretaking Responsibilities by Teacher Experience and Gender 
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Figure 6. Challenges with Technology for Remote Learning by Teacher Experience 
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Figure 7. Changes in Teachers’ Sense of Success Before and After Remote Learning 
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Figure 8. Changes in Likert Response Values of Teachers’ Sense of Success During Remote 
Learning 


Notes: Value labels on bars indicate the change in the Likert response values of sense of success during remote 
learning. The x-axis indicates the percent of teachers reporting each level of change. 
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Figure 9. Changes in Teachers’ Sense of Success by Quintile of Individual Working Conditions 
during Remote Teaching 
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Appendix A: 


Introductory Text for Spring Upbeat Engagement Survey 


Please answer the following questions based on your experiences working at your school PRIOR 
to COVID-19 school closures. We will ask you a set of questions at the end of the survey 
regarding your experience teaching remotely while schools are closed due to COVID-19. 


Remote Teaching Survey 


Please answer the following questions based on your recent experiences teaching remotely while 
schools are closed due to COVID-19. 


Student Engagement 


1. What percent of your students are regularly engaging in learning activities during this 
distance learning period? 


0% to 20% 21% to 40% 41% to 60% 61% to 80% 81% to 100% 


Communication 
2. My district has communicated clear guidelines to teachers about remote teaching in all 


of the phases of this distance learning period. 


3. Administrators at my school have communicated with teachers in a clear and timely way 
during this distance learning period. 
Collaboration 


4. Teachers at my school are collaborating regularly with each other during this distance 
learning period. 


Work/life balance 


5. My school administrators’ expectations about how much work I can accomplish have 
been fair during this distance learning period. 


6. Ihave been able to balance my work with my other responsibilities at home during this 
distance learning period. 


7. Caretaking responsibilities for children and/or dependent adults have made it difficult to 
do my job during this distance learning period. 
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Self-efficacy 


8. Ihave felt successful at supporting my students’ academic development during this 
distance learning period. 


9. Ihave felt successful at supporting my students’ social/emotional well-being during this 
distance learning period. 


10. 1am comfortable using the technological tools required for remote teaching during this 
distance learning period. 


Recognition 


11. [have felt appreciated by my school administration during this distance learning period. 


Resources 


12. My students have had the technological tools they need to engage in virtual learning 
during this distance learning period. 


Professional Development 


13. My district has provided helpful professional development for teaching remotely during 
this distance learning period. 


14. My district has provided helpful professional development for how I can support my 
students’ social/emotional well-being during this distance learning period. 
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Table Al. Sample District Characteristics 


Appendix Tables 


School 
District 


Date of Remote 
State Urbanicity School Learning 
Closure Start Date 


New 
COVID-19 
Cases (per 
Capita) on 
April 20th 


State 
COVID- 
19 Rt 


# of Survey 
Respondents 


Remote Learning Plans 


District A 


District B 


District C 


March 
NY Urban 15th March 23rd 


March 
IL Urban 13th March 16th 


IL, March 
MI Urban 13th March 16th 


18.4 
18.6 


1.56 


2.18 
1.33 


73 


242 


Specific Teacher Instruction: Teachers across county 
were in their respective school buildings Tuesday, 
March 17th to begin remote teaching training and to 
begin rolling out material by March 23rd. 

Delivery Form: Google Classroom and instructional 
packets; District distributed devices to students. 
Grading Policy: Elementary and Middle schools 
switched to a "meets standards"/ "needs improvement" 
grading style. High school students received standard 
grading but had the option of changing courses to 
Pass/Fail. 


Specific Teacher Instruction: Teachers were expected 
to deliver 2 hour lessons to students on Google 
Classroom and to host office hours for students. 
Delivery Form: Google Classroom and instructional 
packets; District distributed devices to students. 
Grading Policy: District implemented Pass/Fail 
policy. 


Specific Teacher Instruction: Beginning March 27th, 
teachers spent Fridays collaboratively planning at- 
home learning lessons. Teachers also facilitated virtual 
morning meetings with their classrooms and held one- 
on-one sessions with their students. 

Delivery Form: Google Classroom, Zoom, Seesaw, 
and printable weekly learning packets; District 
distributed devices to students. 

Grading Policy: Work completed by students could 
only improve their grades. 


March 


District D SC Rural 15th March 17th 

District E VA Rural March March 19th 
13th 

District F SC Suburban’. Match. <a yecon Reh 


but mix 15th 


0.9 


4.1 


1.71 


194 


58 


4,784 


Specific Teacher Instruction: Teachers expected to 
provide students with feedback on instructional 
packets. 

Delivery Form: Google Classroom and instructional 
packets; District distributed devices and also deployed 
Wi-Fi-enabled buses starting March 26th. 

Grading Policy: Work completed during initial two 
weeks of closure were not included in report cards of 
the third quarter. 


Specific Teacher Instruction: March 16th was a 
designated Teacher Workday. This day was taken to 
prepare instructional plans in the case schools were 
closed for an extended period of time. During closure, 
teachers were expected to hold virtual office hours. 
Delivery Form: Instructional packets were available 
every two weeks for pick-up; District distributed 
devices to students. 

Grading Policy: All grading stopped during school 
closure. Final grades represent grades prior to school 
closure. 


Specific Teacher Instruction: Teachers were tasked 
with uploading materials to eLearning sites and to 
have availability for virtual student meetings and calls. 
Delivery Form: Google Classroom and instructional 
packets; District distributed devices to students. 
Grading Policy: Students were mostly graded on 
engagement and participation during e-Learning. 
Engagement is measured by work completion, phone 
calls, Google Meets or any other form of engagement 
with the teacher. 
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March 


District G TX Suburban 19th March 23rd 
District H GA Suburban sett March 16th 
District] VA Rural ee April 13th 


3.6 


5.9 


2.7 


1.75 


2.08 


402 


1,034 


319 


Specific Teacher Instruction: Phase | of the 
Instructional Continuity Plan involved teachers 
posting resources/assignments on the eLearning 
website. Phase 2 included teachers posting 
assignments and instructions on Google Classroom, 
Echo, and other online sources. Unlike phase 1, these 
assignments are not optional. 

Delivery Form: Google Classroom, Echo, and other 
eLearning sources at discretion of schools; District 
distributed devices to students. 

Grading Policy: Grades K-12 will have a 
Pass/Fail/Incomplete grading system. 


Specific Teacher Instruction: Teachers to provide both 
digital and on paper instructional packets for students 
to work on. They will continue to provide assignments 
and accept them until May 15th. 

Delivery Form: Google Classroom and instructional 
Packets; District distributed devices to students. 
Grading Policy: Grades will only be recorded if they 
improve a student's grade. 


Specific Teacher Instruction: Teachers provided 
students with preliminary instructional plans on March 
13th, which included an instructional packet of 10 
days worth of academic content. On March 27, faculty 
members met with principals via ZOOM to discuss 
remote learning. Teachers are expected to 
communicate and provide feedback to students a 
minimum of once per week. 

Delivery Form: Google Classroom and instructional 
packets; District distributed devices to students. 
Grading Policy: Elementary students’ work will not 
be graded; feedback on assignments will be provided. 
Secondary students are encouraged to complete 
assignments. If not, assignments will be made up 
during remediation time during the 2020-2021 
academic school year. 
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District J 


District K 


District L 


March 

VT Rural 15th March 18th 
March 

LA Suburban 13th March 16th 
March 

IL Urban 13th March 16th 


190 


385 


99 


Specific Teacher Instruction: On March 16th, there 
was a delayed two hour opening for students. Teachers 
and staff arrived earlier to begin planning in case of a 
long-term school closure. Under guidance of Agency 
of Education, teachers were tasked to initially plan 
work of "maintenance of education." By April 13th, 
teachers switched to "Continuance of Education." 
Teachers were tasked with creating a planning 
document ahead of April 13th, which had to be 
submitted to their principals. 

Delivery Form: Google Classroom and instructional 
packets; District distributed devices to students. 
Grading Policy: Schools in district released individual 
grading plans. 


Specific Teacher Instruction: Each school developed 
their own requirements and schedule for teachers. 
During phase II of remote learning, which began 
April 20th and April 27th depending on grade level, 
teachers were expected to provide students with 
mandatory work and provide feedback. Unlike prior 
phases, phase HI emphasizes the continuance of 
learning and made all assignments mandatory for 
students. 

Delivery Form: Google Classroom and instructional 
packets; District distributed devices to students. 
Grading Policy: 


Specific Teacher Instruction: Teachers were tasked 
with tracking and monitoring student engagement on a 
weekly basis. 

Delivery Form: Google Classroom and instructional 
packets; District distributed devices to students. 
Grading Policy: Grades entered during remote 
learning can only improve students’ grades. 
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District M IL Urban ia March 18th 18.4 
ar March 

District N IL Urban 13th 18.4 

District O IL Urban see April 13th 18.4 


35 


29 


21 


Specific Teacher Instruction: Each day teachers will 
offer a three hour window check-in for students and 
parents to ask questions about assignments. 

Delivery Form: Google Classroom and instructional 
packets; Districts distributed devices to students. 
Grading Policy: Grading protections in place to 
ensure students with limited access to technology and 
supports are not penalized. 


No information available 


Specific Teacher Instruction: Teachers expected to 
hold office hours for students. 

Delivery Form: Google Classroom and instructional 
packets; District distributed devices to students. 
Grading Policy: For the third quarter, teachers can 
grade and count student work as long as it improves 
the student's grade. No student will receive a fourth 
quarter grade that falls below their third quarter grade. 
If the Q4 grade is below the Q3 grade, students will 
receive a Pass (P). If the student failed their courses, 
they received an incomplete. 


Notes: To find districts’ remote learning information, we first looked at district websites. We then proceeded to search for articles from local news outlets. If 
data on the above indicators were still missing, we checked district Facebook and Twitter accounts. As a last resort, we contacted districts directly via email and 
phone. Dates of school closure announcements refer to the initial school closure announcement made by the state's governor. The Rt scores listed correspond to 
the day of the initial closure announcement. Rt scores indicate how quickly the virus is growing. When Rt is less than 1, this means that the spread of the virus 
is slowing down. Rt scores greater than | mean that the virus is spreading quickly. The number of cases in each county are listed per 100,000 residents. Per 
capita numbers are from April 20th, 2020, which, at the time, was considered a peak day in the United States. All per capita numbers were obtained from 


CovidActNow, while Rt scores were retrieved from rt.live. 
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Table A2. Working Conditions Constructs and Questions by Period 


Construct 


Professional 
Development 


Communication 


Recognition 


Collaboration 


Professional 
Expectations 


Fall Teacher Engagement Survey 


My school offers support to help 
teachers improve. 


The professional development at 
my school has helped me to 
improve my work. 


This year, I've had opportunities 
to learn and develop as a 
professional. 


The expectations for the role that 
I was hired for were made clear 
during the interview and hiring 
process. 

My Principal communicates a 
clear vision for teaching and 
learning for the school. 


The administration at this school 
lets me know when I do good 
work. 

Teachers are recognized publicly 
when they do outstanding work. 
The administration at this school 
notices how hard I work. 

How often do you talk with other 
teachers about best practices in 
your work? 


The work/life balance at my 
school is fair to teachers. 


Spring Teacher Engagement Survey 


My school offers support to help teachers 
improve. 


The professional development at my school 
has helped me to improve my work. 


This year, I've had opportunities to learn and 
develop as a professional. 


The expectations for the role that I was hired 
for were made clear during the interview and 
hiring process. 


My Principal communicates a clear vision for 
teaching and learning for the school. 


The administration at this school lets me know 
when I do good work. 


Teachers are recognized publicly when they 
do outstanding work. 

The administration at this school notices how 
hard I work. 

How often do you talk with other teachers 
about best practices in your work? 


The work/life balance at my school is fair to 
teachers. 


Spring Teaching From Home Survey 


My district has provided helpful professional 
development for teaching remotely during this 
distance learning period. 


My district has provided helpful professional 
development for how I can support my students' 
social/emotional well-being during this distance 
learning period. 


My district has communicated clear guidelines to 
teachers about remote teaching in all of the 
phases of this distance learning period. 


Administrators at my school have communicated 
with teachers in a clear and timely way during 
this distance learning period. 


I have felt appreciated by my school 
administration during this distance learning 
period. 


Teachers at my school are collaborating regularly 
with each other during this distance learning 
period. 


My school administrators’ expectations about 
how much work I can accomplish have been fair 
during this distance learning period. 


33 


Table A3. Longitudinal Growth Analysis of Teachers’ Sense of Success Moderated by Working Conditions Measured at Three Time Periods 


() (2) (3) (4) (5) (6) 

Remote Learning O.148*** — O.152*** — O.LSO*** O14 T#E* — OL148*** 0.14 6*** 

[-29.89] [-28.60] — [-28.97] [-30.33] [-29.98] — [-30.38] 
Spring, in Person 1.712*** = 1.696*** = L.685*** L717  LLT1LO*R* 17 14% 

[13.04] [13.08] [13.03] [13.09] [13.11] [13.10] 
Remote Learning*Individual WC (Fall) 1.061 

[1.29] 
Remote Learning*Individual WC (Spring, in Person) 1.381*** 
[7.21] 
Remote Learning*Individual WC (Spring, Remote) 2.071 *** 
[15.49] 
Remote Learning*Peer-Average WC (Fall) 0.992 
[-0.18] 
Remote Learning*Peer-Average WC (Spring, in Person) 1.028 
[0.58] 
Remote Learning*Peer-Average WC (Spring, Remote) 1.146*** 
[2.90] 

n(Observations) 17,871 17,871 17,871 17,871 17,871 17,871 
n(Teachers) 5,957 5,957 5,957 5,957 5,957 5,957 
Controls Yes Yes Yes Yes Yes Yes 


Notes: * p<.05, ** p<0.01, *** p<0.001. Each column displays results from a separate ordered logistic regression of sense of success regressed on remote 
learning and the interaction between remote learning and working conditions. Estimates are reported as proportional odds ratios. Robust standard errors are 
clustered at the school level and t-statistics are reported in brackets. Teachers’ sense of success is measured by ordinal questions on the Teacher Engagement 
and Teaching from Home surveys. Working conditions are measured by a PCA on a series of questions regarding professional development, communication, 
recognition, collaboration, and professional expectations from the Teacher Engagement and Teaching from Home surveys. See Table 3 for a list of control 


variables. 
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Table A4. Value-Added Models of the Relationship Between Working Conditions Measured at Three Time Periods 
and Changes in Teachers’ Sense of Success 


Fall Spring, in Person Spring, Remote 


Panel A: Individual Working Conditions 


Working Conditions L237 2"* 1.483*** 2.250*** 
[6.33] [10.46] [18.44] 


Panel B: Peer-Average Working Conditions 


Working Conditions 1.070+ 1.104* 1.184*** 
[1.79] [2.54] [4.39] 
n 5,957 5,957 5,957 
Lagged Success Indicators (VAM) Yes Yes Yes 
Controls Yes Yes Yes 
Lagged Working Conditions Indicators No No No 


Notes: + p<.10, * p<.05, ** p<0.01, *** p<0.001. Each cell displays results from a separate ordered logistic regression 
of sense of success regressed on working conditions at each time period. Estimates are reported as proportional odds 
ratios. Robust standard errors are clustered at the school level and t-statistics are reported in brackets. Teachers’ sense 
of success is measured by ordinal questions on the Teacher Engagement and Teaching from Home surveys. Working 
conditions are measured by a PCA on a series of questions regarding professional development, communication, 
recognition, collaboration, and professional expectations from the Teacher Engagement and Teaching from Home 
surveys. See Table 3 for a list of control variables. 


Table A5. Correlation Matrix of Working Conditions Indicators 


SpHne Spans Spring Peer- Spring Peer- 
Fall Individual Individual Individual Fall Peer-Average PMNS pring , 
: ‘ ‘ : Average Working Average Working 
Working Working Working Working oo f ie 
at be : be Fae Conditions, in Conditions, 
Conditions Conditions, in Conditions, Conditions 
Person Remote 
Person Remote 

Fall Individual Working ; 
Conditions 
Spring Individual Working 0.640*** - 
Conditions, in Person 
Spring Individual Working 0.475*** 0.690*** = 
Conditions, Remote 
Fall Peer-Average Working 0:352*2% 0.340*** 0.260*** : 
Conditions 
Spring Peer-Average 0.339%*** 0.369*** 0.305*** 0.883*** - 
Working Conditions, in 
Person 
Spring Peer-Average 0.264*** 0.312*** 0.362*** 0.68 1*** 0.824% - 


Working Conditions, 
Remote 


Notes: *** p<.001. Estimates are Pearson correlation coefficients. Working conditions are measured by a PCA on a series of questions regarding 
professional development, communication, recognition, collaboration, and professional expectations from the Teacher Engagement and Teaching 


From Home surveys. 


